perm filename CONTRO[S86,JMC]2 blob
sn#815658 filedate 1986-04-26 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00012 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00002 00002 contro[s86,jmc] The great Spring 86 AI controversy
C00003 00003 The invitation to the dance.
C00007 00004 The cast of characters.
C00009 00005 The play begins.
C00021 00006 Dreyfus:
C00040 00007 Winograd:
C00051 00008 Searle:
C00074 00009 vijay.ernie@berkeley.edu
C00077 00010 Hofstadter
C00105 00011 Weizenbaum
C00117 00012 Rumelhart
C00130 ENDMK
C⊗;
contro[s86,jmc] The great Spring 86 AI controversy
The invitation to the dance.
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 3 Apr 86 15:14:58 PST
∂07-Feb-86 1529 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 7 Feb 86 15:29:14 PST
Received: by ernie.berkeley.edu (5.44/1.8)
id AA11470; Fri, 7 Feb 86 15:27:39 PST
Date: Fri, 7 Feb 86 15:27:39 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8602072327.AA11470@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Hello Dr. McCarthy,
I, Rajeev Aggarwal, and John Searle are doing a study which
is quite similar to the survey published by Daniel Bobrow in the
AI Journal this year. He will be helping us to make a publishable
version of this study for the AI Journal.
Basically, the whole study can be described/outlined in three stages.
In the first, we have three participants: Hubert/Stuart Dreyfus,
John Searle, and David Rumelhart. They have agreed to provide
approximate 2 page specific criticisms of traditional AI.
(Terry Winograd may also be participating, but this is not certain yet).
In the second stage, four computer scientists actively doing
work in the field will be providing responses to any parts
of the criticisms that they feel need to be refuted, based
on their work, other AI work, or their own philosophies. We
would very much like you to be one of the four participants
in this stage.
All the participants sincerely believe that your presence and views
are very important to such a discussion - for their own benefit and
the various readerships (publications) that we hope will see various
versions of this discussion.
In the last, third stage, we intend to get one last brief
response/comments from the critical side and then a final
statement from the AI researchers.
The exchange of communications will be organized in a manner
so that each participant will have a reasonable amount of time
to respond to other participants, one at a time.
If it is okay with you, we would like to conduct all communication
over the network since this will make the entire study go more
rapidly. We hope you will be able to participate and let
us know soon of your decision. We believe this will be
quite an interesting discussion!
Sincerely,
Vijay Ramamoorthy
The cast of characters.
∂13-Feb-86 1501 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 13 Feb 86 15:00:49 PST
Received: by ernie.berkeley.edu (5.44/1.8)
id AA03983; Thu, 13 Feb 86 14:59:05 PST
Date: Thu, 13 Feb 86 14:59:05 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8602132259.AA03983@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Hello Dr. McCarthy,
Thank you for responding so promptly; The complete list of
participants are John Searle, Hubert and Stuart Dreyfus, David Rumelhart,
Seymour Pappert, Joseph Weizenbaum, Eugene Charniak,
Douglas Hofstadter (in a "middle" position), Terry Winograd,
and yourself.
Next week we will be sending out complete information on
the discussion.
Sincerely,
Vijay Ramamoorthy
The play begins.
∂13-Mar-86 1941 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 13 Mar 86 19:41:18 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA13465; Thu, 13 Mar 86 19:42:23 PST
Date: Thu, 13 Mar 86 19:42:23 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8603140342.AA13465@ernie.berkeley.edu>
To: jmc@su-ai.arpa
***************************************************************
FINALLY --- IT'S HERE!!!
OUR AI DISCUSSION WILL BEGIN NEXT WEEK!!
***************************************************************
Thank you again for your participation, we hope that everyone
will benefit from having the advantage of putting forth their
ideas and receiving responses from such a diverse and dis-
tinguished group of people.
Following is some general information about the written dis-
cussion this study entails:
PURPOSES: We would like this discussion to be a free expression
of ideas about artificial intelligence. It will start with a
series of `critiques' on the traditional approaches that many AI
researchers have, and currently are taking. This will probably be
enough to provoke many responses against the criticisms, and then
responses to those responses. But it needn't always; agreement
is perhaps one of the best things to come out of any discussion,
and we hope that it will emerge in some form from this one. Par-
ticipants will have the consequence of sharpening their positions
and ideologies, and since this is a written discussion, everyone
will have the chance to get at the heart of the beliefs of others
- both by allowing time to think about certain ideas, and by be-
ing able to formulate responses without having to publish them
each time.
We also hope that "this meeting of the minds" will be a testing
grounds for new ideas/hypothesis to gain feedback from others.
There really isn't one sharp line that divides everyone, for al-
most no one agrees completely with anybody else anyway.
FRAMEWORK: There are 3 general stages to this discussion. The
first two will be somewhat formal, with the third being a general
"anything goes" informal exchange. They are outlined as follows:
Stage 1: This stage will consist of some criticisms on
current/traditional AI research; this is basically
to start the discussion; it will be given from group
one of the participants (as we have divided them)
to the other; the each of the criticisms will be
approximately 2 pages.
Stage 2: This stage will be the first response to these criti-
cisms; Each participant from group 2 will have the
opportunity to respond (support/agree or criticize)
anything in each of the critical papers - based on
their research, philosophies, or beliefs. These
responses will then be passed on to the group 1 par-
ticipants.
Stage 3: This last stage will partly build on the first two,
and be supplemented by whatever else comes up. Here
there will be rapid exchanges amongst the various
participants. Everyone will be able to monitor the
the discussion as it progresses.
PARTICIPANTS: This grouping really only applies to the first
2 stages; in the last, it is not important.
Group 1 Group 2
John Searle John McCarthy
Stuart/Hubert Dreyfus Daniel Bobrow
Terry Winograd Seymour Papert
Joseph Weizenbaum Eugene Charniak
In The middle:
Douglas Hofstadter
David Rumelhart
The division was not meant to be a major classification of any
type. It was arrived at based on past stances to traditional
information-processing oriented research. It's only purpose is
to provide part of a knowledge base/foundation for Stage 3.
One note about "In the Middle": for purposes of the first and
second stages, we decided to have Douglas Hofstatder and David
Rumelhart in a position where they will converse with both sides.
TIMETABLE: At the outset, we told everyone that there would be
"a reasonable amount of time to respond." This really applies
to the first two stages, where we would like to keep it to 2
weeks for the production of the first stage, and 2 weeks later
for the responses in the second stage. The third stage will
probably last several weeks, but this is generally open.
The time we have in mind for obtaining the criticisms of stage 1
is... FRIDAY, MARCH 21. At that time, we will pass all of the
papers on to all the group 2 participants. Two weeks from then,
we request all the group 2 responses to be in by FRIDAY, APRIL 4.
These responses will be forwarded to the group 1 members, and the
informal (stage 3) discussion will then begin (probably the most
interesting part). At that point, responses to specific people
will be forwarded immediately to the individuals involved. At
the end of each week, a transcript of the entire week's discus-
sion will be distributed to everyone.
COMMUNICATIONS: The entire discussion, as we have mentioned, will
take place entirely by electronic mail -- the fastest form of
written communication of this sort available to everyone. The
account that will be dedicated to handling all the communications
will be the following:
vijay@ernie.berkeley.edu
Once we start, all information will be processed immediately
after it is received. All messages received will be ack-
nowledged immediately and we hope that everyone will do the same
also. E-mail is reliable, but not "that" reliable.
PUBLICATION: Daniel Bobrow has been kind enough to offer his
help for collating multitudes of responses for publication in the
AI Journal. Furthermore, there will be a neutral introduction
and analysis to the entire discussion.
However, we will also be offering various editions of this dis-
cussion to various prominent national science publications. Our
philosophy here is that noting the quality of articles on AI, it
is clearly better that the current ideas driving AI research be
discussed by those directly involved with it, not by journalists
left to interpret it.
Furthermore, it almost goes without saying that everyone partici-
pating will receive a final copy of the sum total of all com-
munications that go on between the various participants in this
discussion.
Any further questions/problems, please forward them to this
account: vijay@ernie.berkeley.edu
Sincerely,
Vijay Ramamoorthy, U.C. Berkeley (Computer Science)
Rajeev Aggarwal, Bell Laboratories
John Searle, Dept of Philosophy, U.C. Berkeley
Daniel Bobrow, Xerox
(Project Organizers)
P.S. Remember, please acknowledge receipt of this message
through the account you would like us to send all your
responses/coments/information to.
Dreyfus:
∂01-Apr-86 1320 vijay@ernie.berkeley.edu AI DISC: DREYFUS
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 1 Apr 86 13:20:32 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA27558; Tue, 1 Apr 86 13:20:57 PST
Date: Tue, 1 Apr 86 13:20:57 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604012120.AA27558@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: DREYFUS
Hello Dr. McCarthy,
You might have seen a version of the following given by Dreyfus,
however, both Stuart and Hubert Dreyfus say that as a starting
point - it articulates their ideas most clearly.
------------------------------------------------------------------
CONVENTIONAL AI: A DEGENERATING RESEARCH PROGRAM
Looking back over 30 years, the field of conventional
rule-based AI appears more and more to be a perfect example
of what Imre Lakatos has called a degenerating research pro-
gram.[1] AI began auspiciously with Newell and Simon's work
at RAND. In retrospect, we see we failed to appreciate the
importance of this early work. Newell and Simon proved that
computers could do more than calculations. They demonstrated
that the symbols computers manipulate could stand for any-
thing, including features of the real world, and programs
could be used as rules for relating these features, so that
computers acting as logic machines could be used to simulate
certain important aspects of intelligence. Thus the
information-processing model of the mind was born. By 1970
AI, using symbolic representations, had turned into a flour-
ishing research program. Marvin Minsky, head of the M.I.T.
program, predicted that "within a generation the problem of
creating `artificial intelligence' will be substantially
solved."[2]
Then, rather suddenly, the field ran into unexpected
difficulties. The trouble started, as far as we can tell,
with the failure of attempts to program children's story
understanding. It turned out to be much harder than one
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [1] Imre Lakatos, Philosophical Papers, ed. John Wor-
rall, Cambridge University Press, 1978.
$9 [2] Marvin Minsky, Computation: Finite and Infinite
Machines, Prentice Hall, 1967, p. 2.
- 2 -
expected to formulate the required theory of common sense.
It was not, as Minsky had hoped, just a question of catalo-
guing a few hundred thousand facts. The common sense
knowledge problem became the center of concern. Minsky's
mood changed completely in the course of fifteen years. He
told a reporter: "the AI problem is one of the hardest sci-
ence has ever undertaken."[3]
Related problems were also noted although not often
seen as related. Cognitivists discovered the importance of
images and prototypes in human understanding and logic
machines turned out to be very poor at dealing with either
of them. Gradually most researchers have become convinced
that human beings form images and compare them by means of
holistic processes quite different from the logical opera-
tions computers perform on descriptions.[4] Some AI workers
hope for help from parallel processors, machines that can do
many things at once and hence can make millions of infer-
ences per second, but if human image processing operates on
holistic representations that are not descriptions and
relates these representations in other than rule-like ways,
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [3] Gina Kolata, "How Can Computers Get Common
Sense?", Science, Vol. 217, 24 September 1982, p. 1237.
$9 [4] For an account of the experiments which show that
human beings can actually rotate, scan, and otherwise
use images, and the unsuccessful attempts to understand
these capacities in terms of programs which use
features and rules, see Imagery, Ned Block, ed., M.I.T.
Press/Bradford Books, 1981. Also Ned Block, "Mental
Pictures and Cognitive Science," The Philosophical Re-
view, Oct. 1983, pp. 499-541.
- 3 -
this appeal to parallel processing misses the point. The
point is that human beings are able to form and compare
their images in a way that cannot be captured by any number
of procedures that operate on symbolic descriptions.
Another human capacity which computers functioning as
analytic engines cannot copy is the ability to recognize the
similarity between whole images. Recognizing two patterns as
similar, which seems to be a direct process for human
beings, is for a logic machine a complicated process of
first defining each pattern in terms of objective features
and then determining whether, by some objective criterion,
the set of features defining one pattern match the features
defining the other pattern.
As we see it, all AI's problems are versions of one
basic problem. Current AI is based on the idea which has
been around in philosophy since Descartes, that all under-
standing consists in forming and using appropriate represen-
tations. In conventional AI these have been assumed to be
symbolic descriptions. So common sense understanding has to
be understood as some vast body of propositions, beliefs,
rules, facts and procedures. AI's failure to come up with
the appropriate symbolic descriptions is called the common
sense knowledge problem. As thus formulated this problem
has so far turned out to be insoluble, and we predict it
will never be solved.
What hides this impasse is the conviction that the
- 4 -
common sense knowledge problem must be solvable since human
beings have obviously solved it. But human beings may not
normally use common sense knowledge at all. What common
sense understanding amounts to might well be everyday know-
how. By know-how we do not mean procedural rules, but know-
ing what to do in a vast number of special cases. For exam-
ple, common sense physics has turned out to be extremely
hard to spell out in a set of facts and rules. When one
tries, one either requires more common sense to understand
the facts and rules one finds or else one produces formulas
of such complexity that it seems highly unlikely they are in
a child's mind.
Theoretical physics also requires background skills
which may not be formalizable, but the domain itself can be
described by abstract laws that make no reference to
specific cases. AI researchers conclude that common sense
physics too must be expressible as a set of abstract princi-
ples. But it just may be that the problem of finding a
theory of common sense physics is insoluble. By playing
almost endlessly with all sorts of liquids and solids for
several years the child may simply have built up a repertory
of prototypical cases of solids, liquids, etc. and typical
skilled response to their typical behavior in typical cir-
cumstances. There may be no theory of common sense physics
more simple than a list of all such typical cases and even
such a list is useless without a similarity-recognition
ability. If this is indeed the case, and only further
- 5 -
research will give us an answer, we could understand the
initial success and eventual failure of AI. It would seem
that AI techniques should work in isolated domains but fail
in areas such as natural language understanding, speech
recognition, story understanding, and learning where the
structure of the problem mirrors the structure of our every-
day physical and social world.
In 1979 we predicted stagnation for AI, but also
predicted the success of programs called expert systems
which attempted to produce intelligent behavior in domains
such as medical diagnosis and spectrograph analysis which
are completely cut off from everyday common sense. Now we
think we were uncharacteristically over-optimistic concern-
ing the future of intelligent logic machines. It has turned
out that, except in certain structured domains where what
constitutes the relevant facts and how these facts are
changed by decisions is known objectively, no expert system
based on rules extracted by questioning experts does as well
as the experts themselves, even though the computer is pro-
cessing with incredible speed and unerring accuracy what are
supposed to be the experts' rules.
In our just published book Mind Over Machine we attempt
to explain this surprising development. We argue that
beginners in a domain are given principles to follow, but
most domains in which human beings acquire skills and
achieve expertise are, like everyday physics, domains which
- 6 -
do not lend themselves to being understood at an expert
level in terms of principles.[5] Therefore experts, as even
Edward Feigenbaum has noted, are never satisfied with gen-
eral principles but prefer to think of their field of exper-
tise as a huge set of special cases.[6] No wonder expert
systems based on principles abstracted from experts do not,
in unstructured domains, capture those experts' expertise
and so never do as well as the experts themselves.
We still think, as we did in 1965, that someday comput-
ers may be intelligent just as one day the alchemists' dream
of transmuting lead into gold came true. AI may be
achieved, however, only after researchers give up the idea
of finding a local symbolic representation of high-order
macrostructural features describing the world and turn
instead to some sort of microstructural distributed, holis-
tic representation that is directly amenable to association,
generalization and completion. If this is, indeed, the
direction AI should go, it will be aided by the massively
parallel machines on the horizon, but not because parallel
machines can make millions of inferences per second, but
because faster, more parallel architecture can better imple-
ment the kind of neurally inspired processing that does not
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [5] Hubert L. Dreyfus and Stuart E. Dreyfus, Mind
over Machine, Free Press/Macmillan (1986).
$9 [6] Edward A. Feigenbaum and Pamela McCorduck, The
Fifth Generation, Artificial Intelligence and Japan's
Computer Challenge to the World, Addison-Wesley Pub-
lishing Company, 1983, p. 82.
- 7 -
use macrostructural representations of rules and features at
all.[7]
Hubert L. Dreyfus and Stuart E. Dreyfus
University of California, Berkeley
$9←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [7] See for example D. Rumelhart and J. McClelland,
Parallel Distributed Processing: Explorations in the
Microstructure of Cognition, MIT Press/ Bradford Books,
1986.
Winograd:
∂01-Apr-86 1325 vijay@ernie.berkeley.edu AI DISC: Winograd Position
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 1 Apr 86 13:25:24 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA27764; Tue, 1 Apr 86 13:25:53 PST
Date: Tue, 1 Apr 86 13:25:53 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604012125.AA27764@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: Winograd Position
The best thing to do in a short position paper is to put forth some
clear and probably controversial assertions, without giving elaborate
motivations and justifications contrasting them with other ways of
understanding. These fuller discussions appear at length in my recent
book with Fernando Flores, Understanding Computers and Cognition.
1. In characterizing AI, there are two very different starting points.
We can take it as the general enterprise of developing intelligent
artifacts (by any physical means whatsoever), or as the expression of a
coherent methodology and theory.
2. To the extent that AI means "anything anyone might invent that shows
intelligence," discussion belongs in the realm of science fiction, since
there is little concrete to be said. To the extent we are talking about
what people have really done in AI, there is a strong coherent ideology,
variously labelled the "computational paradigm," "cognitive paradigm,"
"physical symbol system hypothesis," etc. Most of the existing AI
enterprise operates within it (including, though to a somewhat lesser
extent, the current work on connectionism).
3. The cognitive symbol-processing approach will have useful
applications, but these will not be as widespread or significant as
proponents claim. In general, those tasks that are closer to
"puzzle-solving" will be best covered, and those closer to "common
sense" and "ordinary understanding" will remain unmechanized. This
applies not only to existing technology, but to any of the foreseeable
improvements following in the general scientific direction that is being
pursued (including "massively" parallel machines, nonmonotonic
reasoning, etc., etc.).
4. I am not so concerned with the danger that attempts to fully
duplicate human intelligence will fail (as long as people don't bank to
heavily on optimistic predictions), but rather that the enterprise has
an effect of redefining intelligence---of shaping human understanding of
what is to count as "intelligent." In particular, AI is based on a
"rationalistic" account of human thought and language, which focusses on
systematic reasoning based on symbolic representations within an
explicitly articulated domain of features. This approach has important
uses, but systematically undervalues other aspects of intelligent human
action, both in the individual and within a tradition. Emphasis on
rationalism is not new to AI, having a long history in Western thought
(beginning with Plato, expressed more thoroughly by Descartes).
Computers (and AI in particular) give it a powerful operational form.
5. A healthy skepticism about AI (and the rationalistic orientation in
general) is needed as a guide for design of computer systems that make
sense. We are easily seduced by the image of the "thinking machine"
into claiming that the problems of designing and working with computer
technology will be solved when the machines get smart enough. The Fifth
Generation hoopla (both the Japanese original report and later books and
responses) is an egregious example of this fallacy. The phenomena of
"computerization" (in its pejorative sense) derive from the
reorganization of social systems to fit the properties of particular
computer implementations. It will not be prevented by having "smart"
machines, and in fact is accelerated by advocating the use of computers
in less structured areas of human life and society.
6. My major interest lies in research (both theoretical and applied)
that will support the development of technology to provide the
advantages of using computers while anticipating and avoiding negative
effects on people's work and lives. The rationalistic tradition does
not provide a sufficient basis for this design, since it takes as its
starting point an impoverished account of what people do. A new
starting point will come from an understanding of the phenomenology of
human communication and use of technology. We can draw on the
philosophical tradition of phenomenology, and its insights can be given
concrete operational meaning in the context of design.
7. It is often claimed that concerns of "social impact" should be left
to the political process, or perhaps to engineers who are directly
developing products, but should be ignored in pursuing "pure science."
These (often self-serving) claims are based on a rationalistic (and
narrow) understanding of science as a human enterprise. They might be
true for some idealized scientist living self-sufficiently and
incommunicado on an isolated island, but are irrelevant to the real
world. The continuing enterprise of any science depends on a public
consensus that supports the allocation of resources to it. This
consensus is maintained by a process of publication and "education" in
which the ideology of the science is promulgated and justified. As
members of the "AI community" we all participate in this, through
writing, talking, and teaching.
8. AI scientists and engineers have a responsibility to take their work
seriously---to recognize that both their inventions and their words have
a serious effect and to consider the effects consciously. The issue
isn't censorship, but positive action. It is useless to try to label
work that "shouldn't be done," but instead we can use our knowledge and
status to advance the things that "should be done," rather than just
those that "can be done." I anticipate a gradual shift of effort and
emphasis within the field as we go beyond the the early science-fiction
dreams that motivated the field, and look at directions for new research
(including theoretical research) that better deals with the realities of
human society. In particular, computers (using AI techniques) will be
understood in terms of the complex and productive ways in which they can
serve as a medium for human-to-human communication, rather than being
personified as surrogate people.
-TERRY WINOGRAD
Searle:
Received: by ernie.berkeley.edu (5.45/1.9)
id AA28407; Thu, 3 Apr 86 15:15:33 PST
Date: Thu, 3 Apr 86 15:15:33 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604032315.AA28407@ernie.berkeley.edu>
To: jmc@su-ai.arpa
TURING THE CHINESE ROOM
John R. Searle
Since various garbled versions of my Chinese room argu-
ment continue to be current in the CS-AI community, I intend
first to set the record straight. Then I intend to review
the current state of the argument concerning strong AI.
Among other things, I am accused of holding the prepos-
terous view that somehow in principle, as a matter of logic,
only carbon-based or perhaps only neuronal-based substances
could have the sorts of thoughts and feelings that humans
and other animals have. I have repeatedly and explicitly
denounced this view. Indeed, I use a variation of the
Chinese room argument against it: simply imagine right now
that your head is opened up and inside is found not neurons
but something else, say, silicon chips. There are no purely
logical constraints that exclude any particular type of sub-
stance in advance.
My actual argument is very simple and can be set out in
a very few steps:
Definition 1. Strong AI is defined as the view that
the appropriately programmed digital computer with the right
inputs and outputs would thereby have a mind in exactly the
same sense that human beings have minds.
It is this view which I set out to refute.
Proposition 1. Programs are purely formal (i.e. syn-
tactical).
I take it this proposition needs no explanation for the
readers of this journal.
Proposition 2. Syntax is neither equivalent to nor
sufficient by itself for semantics.
I take it Proposition 2 is a conceptual or logical truth.
The point of the parable of the Chinese room is simply to
remind us of the truth of this rather obvious point: the man
in the room has all the syntax we can give him, but he does
not thereby acquire the relevant semantics. He still does
not understand Chinese.
It is worth pointing out that the distinction between
syntax and semantics is an absolutely foundational principle
behind modern logic, linguistics, and mathematics.
Proposition 3. Minds have mental contents (i.e. seman-
tic contents).
Now from these three propositions, it follows simply
that strong AI as defined is false. Specifically:
- 2 -
Conclusion 1 : Having a program -- any program by
itself -- is neither sufficient for nor equivalent to having
a mind.
Anyone who wishes to challenge my argument is going to
have to show at least that one of the three "axioms" is
false. It is very hard to see how anybody in the AI commun-
ity would want to challenge any of them. In particular, the
idea that the program is purely formal and the computer is a
formal symbol manipulating device is hardly something that I
need to teach workers in AI .
Once you appreciate the structure of the argument it is
easy to see that the standard replies to it in the strong AI
camp are simply irrelevant because they do not address them-
selves to the actual argument. Thus, for example, the "sys-
tems reply" (according to which `the room,' i.e. the whole
system, understands Chinese even though the man in the room,
i.e. the CPU, does not understand) simply misses the point.
The system has no way of getting from the syntax to the
semantics any more than the man does. The systems reply
cannot evade the sheer inexorability of the syntax/semantics
distinction. Which axioms does it wish to challenge? And
what grounds are being given for the challenge? The "robot
reply" (according to which if we put the system inside a
robot capable of causal interactions with the rest of the
world it would thereby acquire a semantics) simply concedes
that strong AI is false. It admits that syntax would not
be sufficient for semantics but insists that syntax plus
causation would produce a semantics. This involves a
separate mistake that I will come back to, but right now I
want to emphasize that none of the defenders of strong A I
-- a rather large group by the way -- has even begun to make
an effective challenge to any of the three principles I have
enunciated.
That is the argument against strong AI. It is that
simple. Anyone interested only in knowing if strong AI is
false can stop reading right here. But now out of this sim-
ple argument gives rise to a whole lot of other issues.
Some of them are a bit trickier, but I will keep the argu-
ment as simple as possible. As before, the "axioms" must be
obviously true and the derivations must be transparently
valid.
If creating a program is not sufficient for creating a
mind, what would be sufficient? What is the difference
between the relation that mental states have to brains and
the relation that programs have to their hardware implemen-
tations? What are the relations between mental processes
and brain processes anyhow? Well, obviously I am not going
to answer all of these questions in this short paper, but we
can learn a surprising amount by just reminding ourselves of
the logical consequences of what we know already.
- 3 -
One thing we know is this: quite specific neurophysio-
logical and neurobiological processes in the brain ←λc←λa←λu←λs←λe
those states, events, and processes that we think of as
specifically mental, both in humans and in the higher
animals. Of course the brain, like a computer, or for that
matter, like anything else, has a formal level (indeed many
formal levels) of description. But the ←λc←λa←λu←λs←λa←λl powers of the
brain by which it causes mental states have to do with
specific neurobiological features, specific electrochemical
properties of neurons, synapses, synaptic clefts, neuro-
transmitters, boutons, modules, and all the rest of it. We
can summarize this brute empirical fact about how nature
works as:
Proposition 4. Brains cause minds.
Let us think about this fact for a moment. The fact that a
system has mental states and that they are caused by neuro-
physiological processes has to be clearly distinguished from
the fact that a system that has mental states will charac-
teristically behave in certain ways. For a system might
have the mental states and still not behave appropriately
(if, say, the system is human and the motor nervous system
is interfered with in some way) and it might behave in a way
appropriate to having mental states without having any men-
tal states (if, say, a machine is set up to simulate the
input-output functions of the human system without having
the appropriate mental states -- in a familiar example, the
system might emit the right answers to the right questions
in Chinese and still not understand a word of Chinese.) So
the claim that Brains Cause Minds is not to be confused with
the claim that Minds Cause Behavior. Both are true. But
the claim that brains cause minds is a claim about the "bot-
tom up" powers of the brain. It is a summary of the claim
that lower level neurophysiological processes cause, e.g.,
thoughts and feelings. So far it says nothing at all about
external behavior. Just to keep the distinction straight,
let us write this separate proposition as:
Proposition 5. Minds cause behavior.
Now with P. 5, unlike P. 4, we are not talking about bottom
up forms of causation. We are simply summarizing such facts
as that my pain causes me to say "Ouch," my thirst causes me
to drink beer, etc.
From P. 4 and P. 5 by transitivity of causation, we
can infer
Conclusion 2. Brains cause behavior.
But now with the clear distinction between P. 4 & P. 5
and the observation that the input-output relations of human
beings are mediated by mental states, we can see the real
- 4 -
power and implications of P. 4. The causal powers of the
brain consist not merely in the fact stated by C. 2, that
brains causes it to be the case that in response to certain
stimuli a person will emit certain outputs (e.g. someone
pinches me and I say "Ouch"). The claim is rather that
specific biochemical features of the brain by bottom-up
forms of causation cause all of our mental phenomena includ-
ing those mental phenomena that mediate input-output rela-
tions, i.e. those mental phenomena that cause behavior.
(E.g., when someone pinches me and I say "Ouch" it is
because I feel a pain, and the sensation of pain is caused
by neuron firings in the thalamus and the somato sensory
cortex.)
We have then a clear distinction between the causal
powers of the brain to produce mental states and the causal
powers of the brain (together with the rest of the nervous
system) to produce input-output relations. I certainly have
not demonstrated that P. 4 is true, but I take it that its
truth is demonstrated by the past century of neurobiology.
And in any case, does anyone really doubt it? Does anyone
really doubt that all of our mental states are caused by
low level (e.g.neuronal) processes in the brain? Now from
P. 4, it follows trivially that
Conclusion 3. Any system capable of causing minds
would have to have causal powers equivalent to the bottom-up
causal powers of brains.
This is a trivial consequence of P. 4. Conclusion 3 does
not tell us anything about how those causal powers have to
be realized. As far as logical possibility is concerned
they could be realized, as I have pointed out on numerous
occasions, in green slime, silicon chips, vacuum tubes, or
for that matter, old beers cans. I have also claimed that,
as a matter of empirical fact, the probabilities that beer
cans, silicon chips, etc. have the same causal powers as
neurons is, roughly speaking, zero. The chances that chemi-
cal properties of silicon chips will be equal in their
bottom-up causal powers to the properties of neurons is
about as great as the chances that silicon chips will be
able to perform photosynthesis, lactation, digestion, or any
other specifically biological process. However, as I have
said repeatedly, that is an empirical claim on my part, not
something to be established by philosophical argument alone.
But, once again, does anyone in AI really question it? Is
there someone in AI so totally innocent of biological
knowledge that he thinks that the specfic biochemical powers
of human nervous systems can be duplicated in silicon chips
(transistors, vacuum tubes -- you name it)? Frankly, I
doubt it. I think the underlying mistake comes not from
ignorance but from confusion: the confusion is to suppose
that the same input-output function implies the presence of
the same bottom up causation. This view is enshrined in the
- 5 -
Turing test, but a moment's reflection is sufficient to show
that it is false. For example, at an appropriate level of
description an electrical engine can have the same input-
output function as a gasoline engine -- it can be designed
to respond in the same way to the same commands -- but it
works on completely different internal principles. Analo-
gously a system might pass the Turing test perfectly, it
might have the same information processing input-output
functions as those of a human being and still not have any
inner psychology whatever. It might be a total zombie.
We can now see what was wrong with the robot reply. It
had the wrong level of causation. The presence of input-
output causation that would enable a robot to function in
the world ←λi←λm←λp←λl←λi←λe←λs ←λn←λo←λt←λh←λi←λn←λg ←λw←λh←λa←λt←λe←λv←λe←λr about the presence of
bottom-up causation that would produce mental states.
Now from these elementary considerations, we can derive
two further conclusions.
Conclusion 4. The way that brains cause minds cannot
be solely in virtue of instantiating a computer program.
This conclusion follows from Proposition 4 and Conclusion 1,
that is, from the fact that brains do cause minds, and the
fact that programs are not enough, we can derive Conclusion
4.
Conclusion 5. Any artifact that we design, any system
that is created artifically for the purpose of creating
minds, could not do it solely in virtue of instantiating a
computer program , but would have to have causal powers
equivalent to the bottom-up causal powers of the brain.
This conclusion follows from Conclusions 1 and 3.
Now in all of the vast amount of literature that has
grown up around the Chinese room argument, I cannot see that
any of my critics have ever faced up to the sheer logical
structure of the argument. Which of its axioms do they wish
to deny? Which steps in the derivation do they wish to
challenge? What they have done rather, like Hofstatder and
Dennett, is persistently misquote me or attribute views to
me which are not only views I do not hold, but views which
I have explicitly denied. I am prepared to keep winning
this same argument over and over again, because its steps
are so simple and obvious, and its "assumptions" can hardly
be challenged by anybody who accepts the modern conception
of computation and indeed our modern scientific world view.
It can no longer be doubted that the classical concep-
tion of AI, the view that I have called strong AI, is pretty
much obviously false and rests on very simple mistakes. The
question then arises, if strong AI is false what ought AI to
be doing ? What is a reasonable research project for weak
AI? That is a topic for another paper.
-------
-John R. Searle
vijay.ernie@berkeley.edu
responses
I found the three papers disappointingly insubstantial.
I have written out responses to all of them, but I think I'll
hold on to the responses to Searle and the Dreyfus's until
I return from the two week trip to Europe I'm starting on
Sunday. Searle's was the most fun, because it offers the
opportunity to respond to him with the same vigor with
which he treats those with whose opinions he disagrees.
I'm sending you the response to Winograd in the
hopes that it will induce him to overcome his laziness
and subject more of the material from his book to criticism.
Here is the response to the little Winograd wrote.
I would defend the "rationalistic orientation" against the
attack given in Flores's and Winograd's book, which I have read,
had Winograd bothered to present some of the attack. This defense,
however, would have to admit that some of the examples
in the book present problems for previous formalizations used
in AI. Their proper treatment requires a considerable elaboration
of the existing, though new, methods of formalized non-monotonic
reasoning. They may also require something along the lines of
formalized contexts, a subject I have recently been studying.
I especially like the question about whether there is
water in the refrigerator, the issue of what knowledge of flies
may be ascribed to a frog's retina, and the Heidegger (or is
it Flores and Winograd) parable of hammering.
Oh well, too bad.
As for the stuff about considering the consequences of
one's work, one should indeed, but the one must remember that
the scientist isn't the boss of society and can neither force
society to use the results of science nor prevent it from doing
so.
Hofstadter
∂09-Apr-86 1033 vijay@ernie.berkeley.edu AI DISC: Douglas Hofstadter
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 9 Apr 86 10:32:58 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA11863; Wed, 9 Apr 86 10:33:32 PST
Date: Wed, 9 Apr 86 10:33:32 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604091833.AA11863@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: Douglas Hofstadter
**** Please acknowledge receipt of this mailing -Thanks, VR
Impressions of AI
=================
Douglas R. Hofstadter
University of Michigan
April, 1986
I recently went to see an exhibition of Winslow Homer paintings, one
of which was called "Dog on a Log". The title amused me, and I invented
variations on it, such as "Frog on a Dog on a Hog on a Log in a Bog". While
doing this, I was thinking about how people -- especially small children --
love these sorts of simple rhyme, and about how when you hear such things, the
rhyme just jumps out at you, louder than the image behind it. Then I thought
about how speech-understanding programs work, and it occurred to me that if I
pronounced such a phrase into a microphone, a top-notch speech-understanding
program could understand exactly what I had said, in terms of getting all the
words right and putting together the right meaning -- but it would not hear
the rhyme. Not at all. And this seemed to me downright eerie. What would
it be like to hear that phrase PERFECTLY, but not to hear the rhyme? It
would be unimaginably alien -- like being dead, was my feeling.
Many traditional AI people would empathize with my human reaction, and
would say that the program should be fixed up to be able to hear rhymes.
So they would add a "rhyme-detection module", in essence. This does not
seem to me to get you one iota closer to being "alive", however. What about,
say, alliteration? Of course you can add on an alliteration-detection module
as well, and so on ad infinitum. But how many people really believe that this
is how brains work?
When I entered AI some ten years ago, my definition of the field would
have run something like this: "AI is the attempt to understand what thinking
is by making theories, implementing them as computer programs, and learning
from their performance how to improve the theories." Although I wish this
were true, it is far from it. Nowadays, aside from this philosophy, there
are at least two other common views of what AI is: (1) "AI is `knowledge
engineering' -- the field in which one tries to implant expertise in
computers"; and (2) "AI is the branch of computer science concerned with making
computers do more and more things that, for humans, involve intelligence
-- but its practitioners don't care whether the computers accomplish their
results in the way that people do." Both views are highly pragmatic, having
more to do with engineering than with science. Admittedly, excellent minds
can become engrossed in either type of approach, but the concern with the
workings of the human mind has been almost lost, and I believe that the human
mind is much more complex and subtle than anything that people can think
up themselves. I am firmly convinced that AI must first and foremost be
a cognitive science -- a science of the human mind -- rather than a type
of engineering, if it is to make real progress. Artificial intelligence
should be artificial only in the sense that a model of the mind is not as
complex as a real mind.
The engineering approach to building intelligence is a totally open
competition with any idea welcome, whereas in a scientific quest for how minds
work, off-base theories would ideally be pruned by experiments revealing their
flaws. However, since the field is so new, people have not devoted enough
effort to figuring out what constitutes a good test of a theory or of a
program. For instance, I once watched demos of some programs at Yale having
a most baffling property: they used tremendously intricate machinery to
mimic wonderfully flexible acts of cognition -- but those programs worked
with ONLY ONE INPUT. I was reminded of my one visit as a small boy to
Disneyland, where I took thrilling rides down jungle rivers in boats --
but I was most disappointed to find out that these "boats" were actually
running on tracks, so that they had totally fixed routes. (I hope that Roger
Schank would appreciate this reminding-incident.) What should one make of
programs that are so complex, so fancy, so rich with insights -- yet so
brittle? I certainly don't know.
Everybody admits that AI programs don't have common sense yet. One of the
most frequently suggested remedies to this lacuna is that we just give programs
MORE KNOWLEDGE. So one ambitious on-going project is to translate all the
articles in a huge encyclopedia into a sophisticated knowledge-representation
language. I can see the appeal of such a project, but I think it has nothing
whatsoever to do with minds or commmon sense. Adding huge amounts of knowledge
to current "inference engines" is a brute-force approach to a very subtle
problem, and I believe it will fail.
By contrast, I believe that the essence of common sense can be brought out
most clearly in tiny domains, somewhat condescendingly called "toy domains" by
many AI researchers (I like the term). A common view is that AI has exhausted
toy domains and should leave them behind, just as children exhaust the interest
of building blocks and go on to bigger things. I think this is absurd. One
can find huge mysteries in the tiniest of domains, devoid of practically any
world knowledge at all (and for which having vast amounts or world knowledge
would not be of any help).
To my mind, the quality of a scientific project depends critically on
whether it has identified and focused on some central problem in a very clear
way. Toy domains once played (and still could play) the role, in AI, that
idealizations of all sorts have always played in physics. Physicists think
of gases in terms of colliding billiard balls, or solids as perfect lattices;
they think of point particles moving in perfectly straight lines in a perfect
vacuum; and so on. The exact analogue in AI of such idealizations is perhaps
not clear, but certainly it is not a program designed to be an expert in a
highly arcane discipline that practically no one in AI really understands.
Domains like that are so complex that even a weak program can do some things
that people have not thought of. And, although it sounds simple-minded,
I think people are impressed when computers sling around technical jargon in
smooth natural-language discourse, so that weak programs look more awesome
than they are. In a tiny domain, you can't get away with that kind of thing.
Well-chosen toy domains therefore provide much more stringent challenges for
AI than huge domains do.
The canonical toy domain is the MIT "blocks world", home of such classic
AI programs as Guzman's and Waltz's vision programs, Winston's arch-learning
program, Winograd's SHRDLU, and Sussman's HACKER, among others. For reasons
unclear to me, this domain has lost favor in the AI world. People seem to be
under the impression that no challenges of interest could any longer be framed
in such a "small" domain. And yet by no means could any of the above-mentioned
programs be said to have been completed. They all had major defects and acted
very differently from people. They weren't integrated with each other. Simply
producing a blocks-world program that smoothly integrated the skills of ALL
the above-mentioned programs (in improved form) would be a phenomenally hard
task, and, to my mind, a wonderful accomplishment for AI.
As another example of a rich toy domain, I suggest the world of
letterforms -- "a" through "z". Each letter can be written in uncountably
many different ways, and the alphabet as a whole can be designed in uncountably
many different styles. Each letter defines a category that overlaps and rivals
other categories in amazingly subtle ways. Current optical-character-reading
technology has produced useful devices that allow a computer to do pretty well
in reading letters in many styles, but it affords no insight whatsoever into
the fundamental questions of categories, category boundaries, and analogy
(the underpinnings of a uniform visual style). Such devices are as far from
human visual perception as the rhyme-deaf speech-understanding programs I
mentioned above are from human hearing.
The letterform world is an ideal universe in which to study high-level
vision, the interface of perception with categorization, the structure of
very fluid categories, complex associations, overlaps, and rivalries of
categories, and highly abstract analogies, all of which I believe are
at the core of thinking. These issues have only rarely been approached
in traditional AI work -- and yet they are, to me, the problem of mind
in a nutshell. Most of the current work on perception is focused on
low-level (modality-specific) aspects, but I think that the high-level
aspects -- where perception merges with cognition -- are where the greatest
challenge and interest for AI lie. Traditional AI, with its strong focus
on natural language and deduction, tends to presume that all items come
pre-categorized (labeled with words). In such situations, serial models
of thought can do impressive things. On the other hand, such situations
represent but a tiny fraction of what real organisms in the real world
confront in real time.
Recently, there has been a healthy swing away from the serial-cognition
thrust of traditional AI; this movement is often called "connectionism",
although I think "emergent mentality" might be better. Connectionism is
based on the idea that there are fundamental things that serial computers
have shown themselves to be terrible at, and that require a totally different
approach. Years ago, AI people were appalled if you suggested that perhaps
they should be paying attention to how the brain works. They felt that
whole premise of AI was that thought has nothing to do with hardware.
Connectionism is a kind of backlash to that philosophy, and has its roots
in a number of interesting places: neurology, statistical mechanics and
thermodynamics, perhaps automata theory, and so on.
Connectionism's thesis -- that cognition is a collective phenomenon, in
which symbols and representation are statistically emergent phenomena rather
than directly accessible structures and processes -- is a subtle one, which
many old-guard AI people find upsetting, perhaps even inconceivable. Their
resistance is very understandable to me, since I believed in the old view
of AI for a long time. The old view is pretty much a consequence of the
following tight chain of near-equalities:
computation = logic
logic = reasoning
reasoning = thinking
If you believe these premises (and certainly each contains a grain of truth),
then you will have a hard time rejecting the obvious logical conclusion, to
wit:
thinking = computation
This is the basis of standard, mainstream AI. More precisely, that thesis
might be spelled out this way: "Thinking can be implemented as software
that runs on a von-Neumann-type serial machine".
I disagree, in varying amounts, with all three of the "equations" that
underlie this thesis. Actually, the first one is all right, provided you
think of "logic" as meaning merely "digital logic" ("TTL logic"). The third
one is ridiculous, in my opinion, but at least I find it easy to say why: I
have come to believe that most of thinking has nothing to do with reasoning
at all, but with a kind of associationism in which each concept has a hard core
surrounded by a soft blur that cannot possibly be considered part of reasoning.
Typos, speech errors, and many other sorts of error are among the consequences
of that blur; so are clever jokes, beautiful analogies, and intuitive leaps
that result in scientific discovery. The trickiest equation of the three is
the middle one, where the left-hand side ("logic") means two very different
things at once: both "digital (TTL) logic" and "mathematical logic", while
the right-hand side ("reasoning") means "sensible thought pattterns". (This
confounding of meanings of one word is typical of the blurry quality of human
thought -- and in this case it leads to error.) This equation hides perhaps
the subtlest error; because AI people so revere reasoning and because computers
have such a beautifully "logical" internal architecture, one simply WANTS to
equate the two.
All connectionists find fault, for one reason or other, with this
chain of "equations", and believe that the individual micro-elements of
the substrate from which genuine thought (human or artificial) emerges need
not have any representational quality; that that quality attaches only to
large-scale emergent aspects of such a substrate, much as life is a property
of an organism but not of its individual molecules. I am very glad to see
such a school begin to flourish, and I expect many fundamental ideas to
emerge from connectionism over the next decade or two.
My only qualm about this new approach to modeling the mind is that
it so totally eschews AI's original vision of cognition as explicit serial
manipulation of meaning-carrying symbols that it will not provide a framework
in which to naturally address the epistemological questions that AI has been
trying to answer. The point is that AI has always attracted minds that are
interested in how MINDS work, not necessarily how BRAINS work. While it is
probable that mentality is a kind of emergent phenomenon, so that you have
to devote some time to studying its non-thinking micro-components, you have
to beware that you don't get so absorbed in the study of the substrate that
you totally forget about thinking itself.
Connectionist models run the risk of becoming huge simulations that
perform impressively but have little explanatory power. An analogy may
help. Astrophysicists have constructed impressive computer models of the
evolution of galaxies, with thousands of simulated stars interacting via
simulated gravitation. Some of the unexplained properties of galaxies have
been reproduced, such as the formation of spiral arms -- but no one really
understands why. This is a wonderful result, but an explanatory bridge
between the micro- and macro-levels has not been built.
Cognitive psychologists have in essence asked themselves "What is a
concept?" That is, to my mind, the single most important question that AI
ought to be trying to answer. Connectionism may provide an important part
of the answer to this question, but I do not think it will do so alone. Some
insights at the level of thoughts themselves, not just their substrate, will
be essential. Although in some ways, the following will be a shaky analogy,
I would like to suggest it, because it contains a grain of truth. The best
traditional AI (and cognitive psychology) is something like classical physics:
true on a large scale, false on a small scale. Connectionism is something
like quantum mechanics, the ultimate basis of all physics: true on all
scales, but in some sense "irrelevant" on a large scale. The problem
is therefore to link these two vastly different levels. In physics, the
"correspondence principle" says that the equations of quantum mechanics
must turn into their classical counterparts in the limit of large quantum
numbers. In that sense, a beautiful bridge is rigorously established between
the unfamiliar micro-world and the familiar macro-world. I would like to see
such a bridge established between connectionism and the study of cognition
itself, which includes traditional AI, cognitive psychology, linguistics,
and the philosophy of mind. The ultimate goal, to my mind, would be to
provide in this way a hierarchical explanation for such high-level theoretical
constructs as the id, the ego, and the superego (or something along those
lines), allowing us finally to locate in brains or machines the "soft
architecture of the soul".
AI is much more of a hodge-podge than I had expected it to be when I
entered the field. Because it is a fledgling science, and because so much
is unknown, a hundred flowers are blossoming -- but many of them will wilt
rapidly. No one can yet be sure that a particular program or a particular
school or a particular approach is really on the right track to explaining
minds, because most achievements, however grand they may appear on the
surface, are still only tiny fragments of a huge puzzle. It is vitally
important for AI people to resist grandiose claims, and to stress the
limitations of their models. When AI people learn to resist the allure
of complexity, I think they will have taken a major step forward.
Weizenbaum
∂15-Apr-86 2331 vijay@ernie.berkeley.edu AI Disc: Joseph Weizenbaum
Received: from [128.32.0.6] by SU-AI.ARPA with TCP; 15 Apr 86 23:31:29 PST
Received: by ernie.berkeley.edu (5.45/1.12)
id AA14126; Tue, 15 Apr 86 23:32:47 PST
Date: Tue, 15 Apr 86 23:32:47 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604160732.AA14126@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI Disc: Joseph Weizenbaum
About AI
One of the definitions of AI that floats around the computer environment is
that it is the intelligence exhibited by computers when they do things
which, were they done by people, would be considered evidence of the
performer's intelligence. One trouble with this definition is that it is
too broad. It would cover, for example, virtually the entire set of
applications and systems programs ever written. It would, after all,
require considerable intelligence on the part of human beings to solve
systems of differential equations, to invert matrixes, or to manage the
execution of jobs under severe constraints of time, priorities, etc.
Computers do all these things, yet, systems and applications programmers
don't think of themselves as working in AI. Moreover, the "AI community",
a social category on whose definition there is little, if any, consensus at
all, specifically excludes such computer professionals. That this
exclusion is arbitrary can be seen from the fact that literally no one
suggests that the computer systems that routinely land wide body aircraft,
such as Boeing 747's and DC-10's, a task that surely requires great
intelligence on the part of human pilots when they take the controls, no
one suggests that these systems are in the category AI. Ed Feigenbaum,
who, by the way, is emphatically thought not an AI worker by, for example,
the AI insiders at the MIT and Yale Universities' AI laboratories, excluded
MACSYMA, the spectacularly successful applied mathematics system developed
by Joel Moses at MIT concurrent with the developments of Stanford's
DENDRAL, from his list of existing "expert systems" published in his Fifth
Generation, on the ground (so he told me personally) that MACSYMA is not
AI!
Even though the above quasi definition of AI is unsatisfactory in many
repects, it is useful. I prefer it, at least in the present context, to
definitions that appeal to "the way humans do things" or to analogies to
the human brain and such. Under this definition, exercised with a little
bit of common sense and judgment, today's chess playing machines are in,
even though they don't choose (compute) their moves in the same way, or
even in a way resembling the way, chess masters choose their moves. Also
in are robots whose behavior is in part a consequence of their perception
of their environments by means of sensors such as video cameras, touch
sensitive fingers, and so on. Cruise missiles are in. So, of course, is
the set of computer systems developed under the heading "Cognitive
Science".
The reason I prefer to be that all inclusive is that, while I have no
quarrel at all with the idea that there is such a thing as artificial
intelligence, that there exist some, as seen from a performance point of
view, very impressive examples of it, I do not agree with the claim that
"the [computer's] ability to [think and learn and create] is going to
increase rapidly until - in the visible future - the range of problems they
can handle will be coextensive with THE RANGE TO WHICH THE HUMAN MIND HAS
BEEN APPLIED", as Herb Simon wrote almost 30 years ago - in 1958, to be
exact, that is, in the almost no longer visible past.
The artificial intelligencia and I have never disagreed about 1) that
artificial intelligence is possible, 2) that there is no way to put a limit
on the degree - that is, the amount or magnitude - of AI machines may in
future achieve. For all I know, machines may someday be "more intelligent"
than their human models or creators. In any case, I don't know how such an
assertion can be disproved - even apart from the necessary vagueness of its
language. This, by the way, appears to be a point on which Dreyfus and I
disagree. My point of departure from the artificial intelligencia is that I
insist a machine's intelligence, no matter how "great" - must always be
different from and, in certain very important respects, alien to human
intelligence, whereas they, the people claiming to represent AI, believe
that machines can be made to think as humans think, that they can be made
to understand human language as humans do, that, in other words they can
achieve, if that's the right word, an inner life no different in all but
trivial ways from the inner life of human beings.
My belief that machines cannot be made to understand human situations
essentially involved with such matters as respect, love, the psychological
identity (in Eric Ericson's sense, for example) of individuals, and so on,
leads me to the derivative belief that there are areas of human concern to
which AI machines OUGHT not to be applied even if, perhaps especially if,
such machines can be made to ask and answer questions, perhaps in natural
language, in those areas. The machine responses may well create a very
powerful impression, i.e., the illusion, that the machine "understands", it
must, however, reach its conclusions on bases which human beings ought not
to accept in such human problem situations. Machines ought not, for
example, practice clinical psychiatry - so-called talk therapies - nor act
as judges or jurors in human conflict resolution efforts. This conclusion
is not at all sensitive to the manner of achieving AI, that is, whether AI
is realized on Von Neuman machines, connection machines, machines designed
to duplicate the neural (or whatever) structure of the brain. Individual
human beings are the products of their biological constitution and of their
histories. No artifact can have had a history of experiences remotely
resembling that of a human being. Nor can the history of any individual
human being be expressed by a finite bit string (which could be given to a
computer in lieu of its having the human experience itself). I think these
points are obvious. Those who disagree with me on these fundamental points
are free to believe that here, if anywhere, is an outcropping of faith,
theology, belief or whatever, in other words of something that they believe
need not be taken seriously in a discussion of scientific matters.
That the computer is and can be a powerful metaphor, in terms of which much
of the functioning human can be discussed and, in a certain sense,
understood goes without saying in this ninth decade of the 20th century.
Consistent with the position I have here sketched, however, I think there
is more to the life of the mind (at least mine) than cognition. An
understanding of the human mind resting entirely on the computer metaphor
must, in my view, be necessarily grossly incomplete. That it may
nevertheless be useful, even very useful, I don't dispute.
Of those, however, who believe that the improvement in computer chess
playing over the last dozen years or so is a triumph of cognitive science
and not due mainly to the increasing raw power of the computers involved, I
would like to ask what new psychological principles have been discovered
and deployed to account for the strength of todays machine chess play.
-------
- Joseph Weizenbaum
Rumelhart
∂16-Apr-86 2229 vijay@ernie.berkeley.edu AI DISC: David Rumelhart
Received: from [128.32.0.6] by SU-AI.ARPA with TCP; 16 Apr 86 22:28:00 PST
Received: by ernie.berkeley.edu (5.45/1.12)
id AA08440; Wed, 16 Apr 86 22:29:34 PST
Date: Wed, 16 Apr 86 22:29:34 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604170629.AA08440@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: David Rumelhart
(Here is a brief statement.)
Since I suspect different discussants have different conceptions of what
Artificial Intelligence is, it will be worth while for me be as explicit as I
can about my view of the enterprise. AI, like many other fields, can be
separated into three distinct, but interrelated activities. I would
characterize these as (1) theoretical AI, (2) experimental AI and (3) applied
AI. I will sketch below my basic understanding of these three activities.
Theoretical AI
I take theoretical AI to be a branch of mathematics -- a branch devoted
to the development of mathematical systems inspired by considerations of human
cognitive behavior. The activities of the theorist generally involve the
development of formalisms which are thought to be appropriate for expressing
algorithms capable of human-like behavior. Of course, as in all mathematics,
such algorithms and formalisms are not true or false. They are merely useful
or unuseful for some particular task. In this sense AI is not a science any
more than mathematics is a science. There is no empirical domain against
which AI should be held accountable. Of course, psychologists (or AI
practitioners acting as psychologists) may well select some of the
representations and algorithms developed by theoretical AI and state
psychological theories in terms of those formalism. Such theories, like all
scientific theories are then subject to empirical verification. I will say more
about using AI formalisms for stating psychological theories below.
Experimental AI
AI differs from other branches of mathematics in that they do not
ordinarily justify their work on the basis of theorems they might
prove, rather the justification is normally done by the development of computer
programs of one kind or another which are designed to explore and evaluate their
ideas. In this sense, AI is empirical -- do the algorithms do what they were
designed to do? Are there better ones for the kinds of tasks practitioners
have in mind etc? For these reasons computer programs play a unique and
important role in informing and justifying work in AI. They, of course, also
play an important role in giving the AI practitioner a unique perspective on
cognition and therefore, indirectly, shape the kinds of work done by
theoretical AI.
Applied AI
Applied AI is the application of the techniques and formalisms
developed by theoreticians to real problems. There is a fine line here between
applied AI and software engineering in general. AI techniques may be employed
to do whatever tasks computers might be called upon to do. Expert systems are
the most common applications today, but robotics, image analysis, speech
recognition, computer aided instruction and other possibilities are areas where AI techniques can and have been developed.
Cognitive Science
If you are a Cognitive Scientist interested in stating theories
about how people function, it is natural to look to AI as a source for
hypotheses and formalisms within which to state our theories. This, I
think, is because the kinds of mathematics that were developed for the
purposes of expressing theories in the physical sciences have not seemed
to carry over very well to cognitive science. In any case, the
past fifteen years or so have seen an upswing in the use of AI concepts
in accounting for human cognitive processes. However, this does not
necessarily mean that the formalisms that AI has already developed are
the ones which, in the end, will be appropriate or useful. It does
seem clear enough that many of the ideas which dominate modern AI have
been useful in taking a first cut at describing human cognitive
processes. It has seemed to me, however, that many of the ideas from
AI have been a bit misleading and have led people to propose theories
which are probably dead ends. (Note, this does not reflect on
theoretical AI, only on its use for expressing psychological models.)
AI systems have proven to be notoriously brittle and rigid. It seems
to me that this is largely due to the fact that the dominant conception
of computation in AI is based on our understanding of how conventional
computers work. My colleagues and I have become increasingly convinced
that the symbol processing architecture that has served AI more or less
since its inception, probably will not be of as much use in the
description of human cognitive processes (especially those which take
place in fractions of a second) and that we should look elsewhere for
our inspiration for building accounts of cognition. Of course, AI's
tie to symbol processing as the basic paradigm is only historical.
Indeed AI practitioners over the years have considered alternative
computational systems and today a number of AI practitioners are actively
pursuing alternatives.
Parallel Distributed Processing
Parallel distributed processing or PDP for short is the name we give to
the particular brand of "connectionist" models that I see as providing a more
powerful and appropriate formalism within which to formulate models of
cognition. The basic idea is to attempt to give a very general
characterization of what we call "brain style" processing and then to develop
models within that framework. What we have found is that the algorithms most
appropriate for the brain-style architectures are very different than those
appropriate for symbol processing type systems. Moreover, we find that these
algorithms which are natural for PDP models seem better suited as models of
human information processing. They exhibit a number of useful properties
including: ability to rapidly solve a very general "best match" problem,
graceful degradation (rather than brittleness), ability to formulate simple,
powerful learning procedures which let them readily adapt to their environment,
ability to allow a very large number of mutual "soft constraints" to be quickly
and efficiently taken into account in finding the best match, the ability to
naturally represent similarity and automatically generalize to new situations,
naturally form prototypes and a number of other similar features which
seem important in accounting for human cognitive processing. Of
course, the development of these ideas is still in its infancy. It
will probably turn out that some of the features which we now think are
important will not, in the end, turn out to be and features of brains
which we are currently ignoring will turn out to be critical. Still,
we believe that the general strategy of looking directly at brains and
asking about the nature of brain-style processing will ultimately prove
to be a valuable strategy in our understanding of the nature of human
intelligence. Moreover, we suspect that developments in PDP style computation
will ultimately contribute to AI and, to a certain degree, liberate AI from the
historical grip of logic based and symbolic processing systems and encourage
the search for other more appropriate architectures.
D. E. Rumelhart
Institute for Cognitive Science
University of California, San Diego
vijay@ernie.berkeley.edu
Comments on Rumelhart
Copyright 1986, John McCarthy
In general I have only minor quibbles with Rumelhart's classification of
AI related activities. Whether PDP style architectures will win out over
logic based architectures or whether they will be combined is something
for the future to tell us.
vijay@ernie.berkeley.edu
request for acknowledgment
Please acknowledge receipt of the following messages:
general position statement and separate messages on the Dreyfus's, Searle,
Winograd, Hofstadter, Weizenbaum and Rumelhart.